Papers with LM) pretraining
T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition (2021.eacl-demos)
Copied to clipboard
| Challenge: | Language model (LM) pretraining has led to consistent improvements in many downstream tasks, including named entity recognition (NER). |
| Approach: | They propose a Python library for NER LM finetuning that facilitates cross-domain and cross-lingual generalization of LMs finetuned on NER. |
| Outcome: | The proposed library outperforms LMs trained on NERs in cross-domain and cross-lingual generalization tests on nine datasets. |
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for language model pretraining use limited knowledge graph data for knowledge-intensive tasks. |
| Approach: | They propose to make better use of multilingual annotations and language agnostic properties of KG triples for pretraining LMs. |
| Outcome: | The proposed models show significant performance improvements on a wide range of knowledge-intensive cross-lingual tasks. |